891 research outputs found
Stochastic Nonsmooth Convex Optimization with Heavy-Tailed Noises
Recently, several studies consider the stochastic optimization problem but in
a heavy-tailed noise regime, i.e., the difference between the stochastic
gradient and the true gradient is assumed to have a finite -th moment (say
being upper bounded by for some ) where ,
which not only generalizes the traditional finite variance assumption ()
but also has been observed in practice for several different tasks. Under this
challenging assumption, lots of new progress has been made for either convex or
nonconvex problems, however, most of which only consider smooth objectives. In
contrast, people have not fully explored and well understood this problem when
functions are nonsmooth. This paper aims to fill this crucial gap by providing
a comprehensive analysis of stochastic nonsmooth convex optimization with
heavy-tailed noises. We revisit a simple clipping-based algorithm, whereas,
which is only proved to converge in expectation but under the additional strong
convexity assumption. Under appropriate choices of parameters, for both convex
and strongly convex functions, we not only establish the first high-probability
rates but also give refined in-expectation bounds compared with existing works.
Remarkably, all of our results are optimal (or nearly optimal up to logarithmic
factors) with respect to the time horizon even when is unknown in
advance. Additionally, we show how to make the algorithm parameter-free with
respect to , in other words, the algorithm can still guarantee
convergence without any prior knowledge of
Effects of Seawater Corrosion and Freeze-Thaw Cycles on Mechanical Properties of Fatigue Damaged Reinforced Concrete Beams
The effects of seawater corrosion and freeze-thaw cycles on the structural behavior of fatigue damaged reinforced concrete (FDRC) beams were experimentally studied. Results show that the residual strength of FDRC beams reduces as the fatigue load level (the ratio of maximum fatigue load to the ultimate static load) increases. The reduction in the loading capacity of FDRC beams in atmosphere environment was about 6.5% and 17.8% for given fatigue load levels of 0.2 and 0.3, respectively. However, if FDRC beams are exposed to the environment of seawater wet-dry cycles or to the environment of alternating actions of freeze-thaw and seawater immersion, as expected during the service life of RC bridge structures in coastal regions or in cold coastal regions, a more rapid reduction in the strength and stiffness of the beams is observed. The significance of an accurate simulation of working load and service condition RC bridge structures in coastal regions and cold coastal regions is highlighted
Near-Optimal Non-Convex Stochastic Optimization under Generalized Smoothness
The generalized smooth condition, -smoothness, has triggered
people's interest since it is more realistic in many optimization problems
shown by both empirical and theoretical evidence. Two recent works established
the sample complexity to obtain an -stationary
point. However, both require a large batch size on the order of
, which is not only computationally burdensome
but also unsuitable for streaming applications. Additionally, these existing
convergence bounds are established only for the expected rate, which is
inadequate as they do not supply a useful performance guarantee on a single
run. In this work, we solve the prior two problems simultaneously by revisiting
a simple variant of the STORM algorithm. Specifically, under the
-smoothness and affine-type noises, we establish the first
near-optimal high-probability sample
complexity where is the failure probability. Besides, for the
same algorithm, we also recover the optimal sample
complexity for the expected convergence with improved dependence on the
problem-dependent parameter. More importantly, our convergence results only
require a constant batch size in contrast to the previous works.Comment: The whole paper is rewritten with new results in V
Breaking the Lower Bound with (Little) Structure: Acceleration in Non-Convex Stochastic Optimization with Heavy-Tailed Noise
We consider the stochastic optimization problem with smooth but not
necessarily convex objectives in the heavy-tailed noise regime, where the
stochastic gradient's noise is assumed to have bounded th moment
(). Zhang et al. (2020) is the first to prove the
lower bound for convergence (in expectation) and
provides a simple clipping algorithm that matches this optimal rate. Cutkosky
and Mehta (2021) proposes another algorithm, which is shown to achieve the
nearly optimal high-probability convergence guarantee
, where is the probability of
failure. However, this desirable guarantee is only established under the
additional assumption that the stochastic gradient itself is bounded in th
moment, which fails to hold even for quadratic objectives and centered Gaussian
noise.
In this work, we first improve the analysis of the algorithm in Cutkosky and
Mehta (2021) to obtain the same nearly optimal high-probability convergence
rate , without the above-mentioned
restrictive assumption. Next, and curiously, we show that one can achieve a
faster rate than that dictated by the lower bound
with only a tiny bit of structure, i.e., when
the objective function is assumed to be in the form of
, arguably the most widely
applicable class of stochastic optimization problems. For this class of
problems, we propose the first variance-reduced accelerated algorithm and
establish that it guarantees a high-probability convergence rate of
under a mild condition, which is faster
than . Notably, even when specialized to the
finite-variance case, our result yields the (near-)optimal high-probability
rate
Efficient Modeling of Surrogates to Improve Multi-source High-dimensional Biobank Studies
Surrogate variables in electronic health records (EHR) and biobank data play
an important role in biomedical studies due to the scarcity or absence of
chart-reviewed gold standard labels. We develop a novel approach named SASH for
{\bf S}urrogate-{\bf A}ssisted and data-{\bf S}hielding {\bf H}igh-dimensional
integrative regression. It is a semi-supervised approach that efficiently
leverages sizable unlabeled samples with error-prone EHR surrogate outcomes
from multiple local sites, to improve the learning accuracy of the small
gold-labeled data. {To facilitate stable and efficient knowledge extraction
from the surrogates, our method first obtains a preliminary supervised
estimator, and then uses it to assist training a regularized single index model
(SIM) for the surrogates. Interestingly, through a chain of convex and properly
penalized sparse regressions that approximate the SIM loss with
bias-correction, our method avoids the local minima issue of the SIM training,
and fully eliminates the impact of the preliminary estimator's large error. In
addition, it protects individual-level information through
summary-statistics-based data aggregation across the local sites, leveraging a
similar idea of bias-corrected approximation for SIM.} Through simulation
studies, we demonstrate that our method outperforms existing approaches on
finite samples. Finally, we apply our method to develop a high dimensional
genetic risk model for type II diabetes using large-scale data sets from UK and
Mass General Brigham biobanks, where only a small fraction of subjects in one
site has been labeled via chart reviewing
How Does Nuclear Wastewater Discharge Affect Fishery and Marine Environment: A Case Study of Japan
With the increasing use of nuclear energy, human lives have benefited from a variety of aspects since nuclear energy can produce carbon-free electricity. Nevertheless, governments must be cautious about the waste nuclear energy produces for itβs extremely harmful to the environment and has detrimental impacts on human health. Since the nuclear water at the Fukushima plant was released in the following years after 2011, both Japan and its neighboring countries were seriously affected. Some other coastal areas also have varying degrees of pollution depending on the ocean current. The extent of the impact of nuclear wastewater namely the affected areas and the diffusion of elements in nuclear wastewater will be shown in the paper. Additionally, this paper will analyze and elaborate on how nuclear wastewater can affect the marine environment due to the structure of the marine environment and the properties of nuclear wastewater. Lastly, the impact of nuclear wastewater on the fishery in Japan and neighboring countries will be discussed by showing data from relevant research papers. This paper will focus on the impact of nuclear wastewater on the marine environment and the vicinity fishing industry
- β¦